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Abstract This work proposes multi-agent systems setting for concurrent engineering 
system design optimization and gradually paves the way towards examining graph the- 
oretic constructs in the context of multidisciplinary design optimization problem. This 
paper adapts a cooridination strategy based on the well-known nearest neighbor rule 
and corresponding distributed constrained optimization method. The flow of the algo- 
rithm can be described as follow; generated estimates of the optimal (shared design) 
variables are exchanged locally with neighbor subspaces and then updated by comput- 
ing a weighted sum of the local and received estimates. To comply with the consistency 
requirement, the resultant values are projected to local constraint sets. By employing 
the existing rules and results of the field, it has shown that the dual task of reaching 
consensus and asymptotic convergence of the algorithms to locally and globally optimal 
and consistent designs can be achieved. Finally, simulations are provided to illustrate 
the effectiveness and capability of the presented framework. 

Keywords Multidisciplinary design optimization, Consensus algorithms, Projection 
methods, Distributed optimization 



1 INTRODUCTION 



Development and optimization of complex engineering systems arise from the chal- 
lenges of effectively addressing the competing needs of improving performance, reduc- 
ing costs, and enhancing safety. Modern complex engineering systems are usually het- 
erogeneous, highly interconnected and mutually interdependent, both physically and 



through a multitude of information and communication networks Haddad et al. 2006 



Examples of such systems include automobile and rail vehicles design, naval architec- 
ture, electronics, computers, and micro-electro-mechanical systems (MEMS), as well as 
system of systems such as air and ground traffic networks, distributed manufacturing 
environments, and globally distributed supply networks. 
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Design and optimization of these complex engineered systems are multidisciplinary 
in nature, far from optimal, and heavily constrained by both technical and nontechnical 
considerations. It is therefore impossible for one designer, or even a single design team, 
to consider the entire problem as a single design problem. Therefore, the design pro- 
cess is decentralized or distributed over a number of design teams that autonomously 
operates on a single component or aspect of the system. This paper is about Collab- 
orative Multidisciplinary Design Optimization, or CMDO, and deals with concurrent 
optimization of two or more coupled analysis disciplines with distributed computation 
models and numerical optimization. 

Multidisciplinary design optimization [Alexandrov and HussainT 1995| allows de- 
signers to incorporate all relevant disciplines simultaneously. The simultaneous opti- 
mization is superior to the commonly used sequential design methods, since it can 
exploit the interactions between the disciplines. Design is both analysis and synthesis, 
and is compromise in the balance of conflicting requirements. However, finding the 
best compromise by including all disciplines simultaneously significantly increases the 
complexity of the problem. 

During the past three decades, decomposition-based design optimization strategies 
|Sobieski and Haftka 1997) as a natural approach have drawn a great deal of atten- 
tion of researchers to solve the design problem of complex systems in a distributed 
way. However, most works in such decentralized design optimization settings address 
a hierarchical or sequential evaluation of a master problem and some disciplinary sub- 
problems. The optimal set of design variables for the upper level becomes the objec- 
tive/constraints responses for the lower level. At each level, disciplines are optimized 
separately. Accordingly, two general methods are elaborated in literature; single-level 
methods, which have centralized decision-making authority and do not allow design 
decisions to be made at the disciplinary level, and multi-level methods, which a central 
master optimization problem is introduced to coordinate the interactions between the 
disciplinary sub-problems. In some multi-level methods, such distributed authority for 
decision making is sometimes referred to as distributed design optimization schemes. 
However, in this paper, by distributed methods we mean the same decentralized deci- 
sion making capability without any master or central coordinator. 

However, several recently proposed distributed methods are placed in the multi- 
level category. Typically, these methods hierarchically decompose the underlying design 
problem into sub-systems along the lines of systems, subsystems, and components or 
usually partition in a non-hierarchical fashion along disciplinary lines. A review of 
single-level and multi-level methods can be found in [Cramer et al. 1994| and [Tosser 



ams et al. 2009 



Several partitioning and coordination methods have been proposed for Multidisci- 
plinary Design Optimization (MDO) problem including linear decomposition method 
(OLD) |Sobieski 



1982 



Concurrent Subspace Optimization (CSSO) |Sobieski 



BLISS [Sobieski et al. 2003 , analytical target casc ading (ATC) |Michelena et at 
Quasiseparable decomposition (QSD) approach 



Haftka and Watson 



2005 



1988 



1999 



penalty 

decomposition formulation |Demiguel and Murray 2006), Augmented Lagrangian De- 
composition method for Quasi-separable problems |Tosserams et aL 2007| , and Col- 
laborative Optimization (CO) Braun 1996| , Enhanced Collaborative Optimization 
(ECO) " 



Roth and Kroo 2008 



A full description of decomposition and coordination 
strategies in multidisciplinary design optimization problems is beyond the scope of this 
paper and was already reported in [Tosserams et al. 2009] . As mentioned before, most 
of these approaches are essentially decentralized in the way that they organize a fully 
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centralized problem in a hierarchical (i.e., sequential) structure. It is clear that we have 
reached the limits of what these approaches can do |Allen et al. 2011 . To proceed, 
we need a more rigorous and deeper understanding of complex engineered systems and 
how they should be designed. 

The penalty relaxation methods 



Michelena et al.| |1999| , |Blouin et al] |2005 



Tosscrams et al. 2008| ) relax the coupling constraints of MDO problems to arrive 
at subproblems with separable constraint sets. ALC jTosserams et al.j|2008] provides a 
flexible coordination structure, not necessarily hierarchical that uses penalty relaxation 
methods in tandem with algorithms for solving systems of equations. The convergence 
to KKT points of the original problems is the main advantage of penalty methods. 
Some penalty relaxations methods have only been developed for quasi-separable prob- 
lems coupled through a set of coupling variables; coupling objectives and constraints 
are not allowed. In jTosserams et alT 



2006 , an Augmented Lagrangian Relaxation 



for Analytical Target Cascading using the Alternating Directions Method of Multipli- 
ers are proposed. In [Tosserams et aL 2008) , a new penalty relaxation coordination 
method is proposed that can be used to solve MDO problems with coupling variables, 
a coupling objective, and coupling constraints. However, ALC results in a very large 
number of consistency constraints; only a subset is actually required to ensure con- 
sistency. On the other hand, such methods rely on excessively large penalty factors 
for sufficiently accurate solutions. Several consistency constraints allocation guidelines 



have been proposed for ALC implementations in Allison and Papalambros 



Our work is also related to game-based design approaches 



2010 



Hernandez et al. 



2002 



|Xiao et al. 2005 |Ciucci et al. 2012 . The pioneer work of Lewis et al [Lewis and 



Mistree 1997 in the last nineties suggested a game theoretic approach to model inter- 



actions in multidisciplinary design. In (Hernandez et al. 20021, the authors investigated 



a quadratic and eigen-based formulation to enhance convergence speed. They conclude 
that passing more information generally leads to convergence to a Pareto-optimal set. 
This work is built upon the constrained consensus method |Nedic and Ozd aglar 



2009| |Nedic et al. 2010 , adapts a distributed computation framework for general 



complex system design and optimization problem and gradually paves the way to- 
wards examining graph theoretic constructs in the context of multidisciplinary design 
optimization problem. Convergence properties of the agreement protocol can be proved 
using existing result on algebraic graph theory |Godsil and Royle 2001] , in particular, 
spectral properties of the underlying graph. 

This paper considers the design problem of complex engineering systems, consisting 
of several (including both local and global) objectives, design variables and constraints 
corresponding to different disciplines and assigned to several teams. In other words, 
system- wide disciplines (i.e., those are spread throughout the problem space) are also 
allowed. In particular, these distributed multi-objective multidisciplinary design op- 
timization problem is presented in a multi-agent setting, providing necessary infor- 
mation passing structure to come up with an appropriate decision. In brief, the flow 
of the proposed algorithm can be described as follow; generated estimates of shared 
design variables and optimal linking variables (if any) are exchanged directly among 
subspaces. Afterwards, each agent computes a weighted sum of its estimates and re- 
ceived estimates. Then, each agent projects the variables received from other agents to 
its constraint sets to maintain consistency. 

Analysis shows that the performance of the algorithm largely depends on the sub- 
problems as well as the network structure and communication protocols. Afterwards, 
our main focus is on developing distributed algorithms that guarantees asymptotic 
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consensus on the shared and linking variables while maintains the feasibility with re- 
spect to the constraint sets. The proposed multi-agent framework is adapted to flexibly 
address both system-level (global) and discipline-level (local) issues, without any re- 
quirement of objectives and constraints relaxation. Moreover, it has been shown that 
the proposed framework is in great accordance with the corresponding design and de- 
velopment teams. Finally, the main feature that distinguishes the proposed method 
from others is its structural flexibility, design autonomy, and rigor mathematical and 
graphical representation. We believe that the most challenging part of the method (es- 
pecially for general design optimization problem) is the implementation and analysis of 
its nonlinear part; i.e., projection algorithm. In addition, coupling between subspaces 
might be another challenging issue and require further investigations both in theory 
and in experiment. Attention is focused in this paper on the general case, and some 
companion papers will be devoted to special cases that allow further analysis. 

The rest of the paper is organized as follows. Section II is devoted to provide the nec- 
essary mathematical and graph theoretical foundations and computational algorithms 
used for incremental coordination and projection. Section III introduces the main prob- 
lem we consider, discusses the assumptions made in the proposed model, and presents 
our algorithm. Section IV presents the performance analysis and the convergence issues 
of the algorithm, followed by section V that demonstrates the algorithm and compares 
the results with that of the well-known All-in-once ( AIO) method. Section VI concludes 
the paper. 



2 PRELIMINARIES 

In this section, we discuss the standard consensus algorithm and the constrained con- 
sensus algorithm. Some of the equations and ideas for this section covering consensus, 
projection and constrained consensus algorithms, originate from |Nedic et al. 2010] , 
demonstrating the existing strong theoretical foundation. They are included here to en- 
sure a complete statement of the problem at hand and an accurate comparison between 
coordination strategies. 



2.1 Consensus Algorithms 



Distributed average consensus algorithms are a class of iterative update schemes that 
work based on the neighbor interactions. In recent years, there has been a surge of 
interests in distributed computing methods based on the average consensus algorithms 
Jadbabaie et al. 2003 |01fati-Saber and Murray 2004 , and it has found applications 
in rendezvous, formation control, flocking, attitude alignment, decentralized task as- 
signment, and sensor networks. Let's consider a network of i agents, represented by 
V = {1,2, ... ,n}. The neighbors of node i is a set of nodes j £ V where communi- 
cating with node j through a directed link e = At each time k + 1, we assume 
that agent i receives information Zj (k) from neighboring agents j and updates its esti- 
mate by adding a weighted sum of the local discrepancies, i.e., the differences between 
neighboring node values and its own. In Olfati-Saber and Murray 



2004 , Olfati-saber 



and Murray show that the following linear dynamic system: 
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2i(fc + l) = Zi{k) + J2 ai 3 {k){ Zj (k) ~ Zl (k)) (1) 

where a,ij(k) is a weight associated with the edge € iVj and fc = 0, 1, . . . , 

solves a consensus problem. More precisely, let z^'s be n constants, then with the set of 
initial states Zi(0) = Zj, the state of all agents asymptotically converges to the average 
value z = j^y^Zj provided that the network is connected. 

i 



2.2 Projection 

Let z be an element in a Hilbert space H and let Z be a closed (possibly non-convex) 
subset of i/. We use Pz[z] to denote the projection of a vector z onto a closed convex 
set Z, and define as follows: 



z[ z l — argmin ||z — z\\ 



There is always at least one such point for each z, namely where H is a finite 
dimensional Hilbert space. If Z is convex as well as closed then each z has exactly one 
projection point Pz[z] |Luenberger 1969| . 



2.3 Constrained Consensus Algorithm 



The constrained consensus problem is to achieve asymptotic consensus on the local 
decision variables, Zj, through information exchange with the neighboring nodes in the 
presence of t he constraint sets, Zj G Zi. A distributed algorithm for this problem was 
proposed in Nedic et al. 2010 . In the algorithm, i th local variable zi(k) is executed 
as follows: 



i {k + l) = P Zi ^a ij {k + l)z j {k)] 

3=1 



and it can also be written: 



(2) 



Vi (k + 1) = Zi(k) + Y, a ij( k + !)( 2 #) ~ ^0)) 

3=1 

z i (k + l) = P Zi [v i (k + l)] 



(3) 
(4) 



Illustration of the algorithm in 2D case is given in Figure [T] The constrained con- 
sensus algorithm has several advantages over alternating projection method, the most 
significant being that concurrent computation of subspaces is possible. Accordingly, in 
a distributed setting, internal dynamics of subspaces are coupled together, cooperating 
(or may be competing) to achieve the overall system objectives: 
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Fig. 1 Constrained consensus algorithm 



Vi(k + 1) = (1 - <Hj(k + l))zi{k) + a t;j (k + l)zj(k) (5) 
Zi(k + l) = P Zi [vi(k + l)} (6) 



where i / j, i, j = 1,2. 



2.4 Constrained Optimization 

Distributed constrained optimization discusses the problem of optimizing the sum of 
convex objective functions corresponding to m connected agents. The goal of the agents 
is to cooperatively solve the constrained optimization problem 



X> (*) (7) 



subject to z 6 (1 

i—l 



where the local objective function of agent i, fi : R n — ¥ R, is a convex function, 
and the local constraint set of agent i, Z{ C W 1 , is a closed convex set that are known 
to agent only. In [Nedic et al.|[20To] , the following distributed projected subgradient 
algorithm are proposed for solving problem Q 



Vi (k) = (*) *j (*) (8) 

Zl (fc + 1) =p z . [ Vi (fc) - aud, (*)] (9) 

where the scalars ajj (fe) are nonnegative weights and the scalar > is a stepsize. 
The vector di (fc) is a subgradient of the agent local objective function fi (z) at z = 
Vi (fc). 
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3 DISTRIBUTED MULTI-OBJECTIVE MULTI-DISCIPLINARY 
DESIGN OPTIMIZATION 

In this section we introduce the main problem we consider. We also discuss the assump- 
tions made in our model and propose our algorithm. The general formulation for this 
section covering multidisciplinary design optimization is in the spirit of the framework 
outlined in design optimization literature jAlexandrov and HussainT] |1995) . 



3.1 MDO Problem Formulation 



The optimization problem usually encountered in many engineering system design is 
considered to be a nonlinear programming problem. The general multidisciplinary de- 
sign optimization (MDO) problem is formulated as follows: 



min f(x,y,z) 

z=\z l ,z s j 

s.t. g (x, y, z) < 
h(x,y,z) = 

V«, j j= i,Vi = Cji (xj,yj,Zj) (10) 
Vi, x-i — (x{, yi : Zi) 



where 



z = (zi, Z2, ■ ■ ■ , z m ) denote design vector, consist of design variables from dif- 
ferent disciplines. Moreover, these variables can be partitioned into shared 
and local design variables of ith subspace; zf and z\, respectively. 

x = (x\, X2, ■ ■ ■ , Xm) is state vector and depend on both linking and design 
variables. 

j/j's denote linking variables and provide coupling among different subsystems. 

/, grand h are vector- valued objective function, inequality and equality con- 
straints functions, respectively. 

Cji : {x i ,y i , Zj) — > y^ denote the coupling function from the subsystem i to the 
subsystem j. 

T{ : (x i ,y i ,z i ) — > x i denote state transition functions that compute state 
variables of subspace i. 



We assume that the local objective function /j and the local coupling Cji and transition 
Ti functions are known to agent only. Recall that z* is a local minimum of ( 10 1 if there 
exists e > such that f(x, y,z*) < f{x, y, z) for all z G Sf] B(z* , e) and corresponding 
state and coupling variables, where S is the feasible region. 

The optimal design of complex engineering systems involves concurrent optimiza- 
tion of several objectives, constrained by both local and global issues. A distributed 
variant of equation ( 10 1 can be represented as follows: 



s 
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min [fo(x,y,z);fi(x,y,z);...;f m (x,y,z)] (11) 

z=(z l ,z s ) 

s.t. gi {x, y, z) < for all i 
hi (x, y, z) = for all i 
Vi,i =£i,yi = cji (xj,yj,Zj) 

This problem can be formulated in a more compact form, which makes it suitable 
for representation of distributed multi-objective design optimization problems. Let S 
denote the feasible region of {n), i.e., 

Si = {(a:' G X u y 6 Y h z G Z,) : 

9i(x ,y , z )<0, hi(x ,y ,z) = o| , i = 0,1, ...,m 

Accordingly, we use (pi to represent functional evolution of the state and coupling 
variables of subspace i: 



<Pi = {(xi G Xi,yi G Yi,Zi G Z<) : x l = T^x^yi, z,-), 

and then let 



5= f|5, ; 



i=l 



Then the problem (111 can be written: 



minF (a;, y, z) = [fo(x, y, z); fi (x, y, z); . . . ; f m (x, y, z)] 

z^Z 

<p(x,y,z)eS (12) 
The goal of the agents is to cooperatively optimize both local and global objectives: 



m 

f(x)=f (x) + ^2f i (x) 
i=i 
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Fig. 2 Digraph represent structure of design optimization problem 
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Fig. 3 A typical structure for information-passing; (left) Distributed constrained optimization, 
(right) Distributed design optimization; shared variables (green), coupling variables (red), and 
state variables (black) 



3.2 Distributed Computation Model 



In multiobjective multidisciplinary design problem, interconnections and interactions 
among subspaces are usually complex. In such complex problems, information- passing 
among subspaces and structural organization of the subproblems is as important as sub- 
spaces optimization. In the following, we discuss the proposed distributed computation 
model from a graph theoretic perspective to handle information passing among sub- 
problems. Moreover, an analysis section is devoted to discuss basic assumptions and 
corresponding results from algebraic graph theory, which provides analytical frame- 
work, required for analysis of coordination strategy and communication protocols. A 
detailed discussion of this topic is beyond the scope of this paper and a brief overview 
of the subject is given. For further information on this topic, readers are referred to 
Ncdic et al. 2010 . Recall we represent the structure of our design optimization prob- 
lem, consists of m subproblems by undirected graph G = (V,E). A typical structure 
of the design problem is illustrated in Figure [2] 

In MDO problems, there are at least two different types of information being ex- 
changed; shared design variables and coupled variables. Figure [3] illustrates a typical 
flow of information among subspaces. 

In addition, communication protocols play a critical role in providing effective in- 
formation sharing between several design teams. A communication protocol consists of 
a set of rules which govern the orderly exchange of information among entities. Two 
general classes of protocols are proposed; synchronous and asynchronous protocols. 
Gossip communication protocol of |Kempe et al. 2003] and the broadcast protocol of 
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Fig. 4 Illustration of the proposed algorithm 



|Aysal et al. 2009) are some examples of these protocols. For a detailed review of this 
topic see [Aysal et al7| |2009| . 



3.3 Collaborative Multidisciplinary Design Optimization (CMDO) algorithm 

We next consider the problem of general multi-objective multi-disciplinary distributed 



design optimization (M2DO) (111 corresponding to m agents connected over a fixed 



topology. We also introduce two different implementations of the proposed method 
for solving this problem. Recall the local design variable associated with subspace i 
is denoted by Zi(k) and is restricted to lie in a local constraint set Z^. In the multi- 
step version of the algorithm, each agent i starts with an initial estimate Zi(0) £ Zi 
and update its estimate by combining the estimates received from its neighbors, by 
taking an optimization step to minimize its objective function. Then she/he exchanges 
this estimate with neighbor subspaces. Finally, the resulting value is projected into 
local feasible set. This algorithm illustrates the intuition behind the proposed method 
and allows employing heuristics in projection and optimization steps. In interleaved 
version, the optimization step is augmented in the constrained consensus algorithm. 
We consider this formulation when discussing performance of the algorithm. 

Although there are a lot of choices for projection or other steps, but we do not 
discuss it in this paper. However, the main advantage of this algorithm lies in the fact 
that it is possible to independently specify each step. 

An illustration of the algorithm is presented in Figure [4] In the following sections, 
we discuss the behavior of this algorithm. 



4 PERFORMANCE ANALYSIS 



We next discuss the behavior of the proposed algorithm. Analysis of the general MDO 
problem is difficult because of the coupling among subspaces and internal state tran- 
sition. In this paper, we focus on some special cases including quadratic objective 
functions with linear subspaces. Firstly, some existing results in the field are discussed. 
Then, a class of multidisciplinary design optimization problems are defined and some 
extra assumptions are placed on information structure and coordination strategy that 
can guarantee the convergence of the algorithm. 
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Algorithm 1 Multi-Step Algorithm 

Algorithm 1: Multi-step version This algorithm is sequentially executed in three steps as 
follows: 

Update Let's consider the initial estimate of the local and shared design variables z'(0) £ 
Z',z|(0) £ Zf , respectively. Hereinafter, for the sake of notational simplicity, we omit the 
super-index s in z?. These estimates Zi, are exchanged directly among neighbor subspaces 
according to the problem structure. Each agent updates its estimate by forming a convex 
combination of the local estimate of shared design variable and that of neighbor subspaces. 
This update mechanism at time tj. is formally stated by the following equation: 



m 

Zi(k + 1) = zi(fc) + ai (k)J2 a ij( k + l)C*iW - z A k )) (13) 

3=1 

where (fc)determines information passing among subspaces. The coupling and state vari- 
ables are also updated as follow: 

i,j¥ z i,yi=Cji(x j ,yj,z j ) (14) 
Xi = X 1 (y i ,z i ) (15) 

At the end of this step, agents share their estimate of shared values and coupling variables 
among each other. 

Optimization Afterwards, each agent sets the updated shared design variable, Zi{k + 1) as 
an initial value, updates his/her objective based on the recent local state variables and 
received coupling variables and then takes an optimization step according to the following 
optimization problem minfi(xi,yi, Zi). Let's denote the resulting optimal design variables 

at time f(fc_|-i) by z *{ k + 1)- 
Projection The last optimization step may result in values outside the feasible region of each 
subspace. Therefore, we project the resultant value into the feasible region to satisfy con- 
straints 

**(* + !) = P S , [*?(* + !)] (16) 



Algorithm 2 Interleaved Algorithm 

Algorithm 2: In this algorithm, the optimization step is augmented in the constrained con- 
sensus algorithm (f2p as follow 



Zi (k + 1) = P Sl 



a i ( k ) ^2 a ij ( k ) z 3 ( k ) ~ Ti ( k ) d i (k) 



(17) 



Where the vector di{k + 1) is an optimization step of the objective function of subspace i, i.e.. 
fi at point (xi(k),yi(k), Zi(k)), updated at each time step t^. 



4.1 Constrained Consensus 



This section provides a summary description of the existing convergence results for 
distributed optimization. We adopt the following assumptions in our analysis following 
Blondel et al. |Blondel et aT| |2005| and |Nedic et al^||2010| . 



Assumptionl: (Network Topology) We assume information passing among sub- 
spaces takes place at discrete time steps, t^. Recall the weight associated with the 
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edges in network graph can be denoted by ay(fc), or where A(k) is called the 

network weight matrix. 

Assumption2: ( Weights Rule) There exists a scalar r\ with < n < 1 such that for 
all i,j € {1, m}, and k > 

1. a u (k) > rj 

2. aij(k) > r) when subspace j communicates with subspace i, and aj,(fc) = other- 
wise. 

m 

3. ^ay(fc) = l(ro«) stochastic) 
i=i 

Assumption3: (Connectivity and symmetry) The graph (V, J5oo) is connected, where 
Eoo is the set of edges (j, i) representing subspace pairs communicating directly in- 
finitely many times, i.e., 

Eoo = {[j>i)\(j,i) £ Efc for infinitely many indices A:} 

Moreover, the graph (V, -Eoo) is symmetric, i.e., the weights satisfy a^j(k) = aj^(k) 
for all i,j. 

Assumption^ (Bounded intercommunication intervals) If i communicates to j an 
infinite number of times [that is, if (i, j) £ E(t) infinitely often], then there is some B 
such that, for all t, 



(i,j) <E E(t) U E(t+ 1) U •• • U E(t + B - 1) 



Assumption5:(.DoHWy Stochasticity) The vectors 
Oi(Jfe) = (a a (k), . . . ,a lm (k)) satisfy: 

m 

a) ai(k) > and ^ a ij(k) = 1 for all i and fc, i.e., the vector aj(fc). 

t=l 

m 

b) ^ a>ij(k) = 1 for all j and fc. 
i=i 

Remark 1 This assumption establishes that each agent takes a convex combination of 
its estimate and the estimates of its neighbors. Moreover, second part of this assumption 
together with Assumption 2, ensures that the estimate of every agent is influenced by 
the estimates of every other agent with the same frequency in the limit, i.e., all agents 
are equally influential in the long run |Nedic et al. 2010| . 



Lemma 1 'Olfati-Saber et al. 2007j Let G be a connected undirected graph. Then, the 
algorithm fw asymptotically solves an average consensus problem for all initial states. 

Remark 2 Simply, this lemma states that if there exist some paths for flow of informa- 
tion, consensus is eventually achieved. Therefore, information exchange among different 
subspaces with the same shared design variables are necessary for convergence. 



Proposition 1 



Nedic et al. 



h Zi be nonempty. 



20101 (Consensus) Let the set Z 

Also, let Weights" Rule, Doubly Stochasticity, Connectivity, and Information Exchange 
Assumptions hold (cf. Assumptions 2, 3, 4, an d 5). For all i, let the sequence {zi(k)} 
be generated by the constrained consensus algorithm We then have for some z G Z 
and all i, lim \\zi (k) — 5|| = 0. 
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4.2 Distributed Constrained Optimization 

In this section, we discuss the existing results for the constrained optimization problem. 
Let's first consider the following assumptions. 
Assumption 6: {Same Constraint Set) 

— The constraint sets Z^ are the same, i.e, Zi = Z for a closed convex set Z. 

— The subgradient sets of each /j are bounded over the set Z, i.e., there is a scalar 
L > such that for all i 

\\d\\ < L for all d G dfc (z) and all z G Z 

Assumption 7: [Compactness) For each i, the local constraint set Zi is a compact set, 
i.e., there exists a scalar B > such that 



Izll < B for all z G Zi and all i 



The next proposition presents convergence result for the same constraint set case. 
In particular, it is shown that the iterates of the projected subgradient algorithm (JsJ>- 
(JqJ) converge to an optimal solution when we use a stepsize converging to zero fast 
enough. 



Proposition 2 \Nedic et al. 2010 ] Let Weights Rule, Doubly Stochasticity, Connec- 



tivity, Information Exchange, and Same Constraint Set Assumptions hold (cf. Assump- 
tions 2, 3, 4, 5, and 6). Let be the iterates generated by the algorithm ^-^ty 
with the stepsize satisfying ^ fe = oo and y\ otf, < oo. In addition, assume that the 
optimal solution set Z* is nonempty. Then, there exists an optimal point z* G Z* such 
that 



lim \\zi (k) — 2*|| =0 for all i 
k— > oo 

The next proposition presents convergence result for the projected subgradient algo- 
rithm (|8|-|9| in the uniform weight case. 



Proposition 3 \Nedic et al. 20101 Let Interior Point and Compactness Assumpt 



ions 



holdjcf. Assumptions 1 and 7). Let {zj(fc)} be the iterates generated by the algorithm 
{<§])- with the weight vectors a,; (fc) = (1/m, . . . , 1/m) for all i and k, and the stepsize 
satisfying X/fc a fc = 00 an d Ylk a 'k- 00 ■ Then, the sequences {zi(k)}, i = l,...,m, 
converge to the same optimal point, i.e. 



lim 2j (fc) — z* =0 for some x* G X* and all i 

k—^oo 
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4.3 Collaborative Multidisciplinary Design Optimization 

In this section, we present our results. We consider a distributed multidisciplinary 
design optimization problem with quadratic objectives and linear constraints. In a 
special case, we establish conditions under which convergence of the algorithm (171 to 
optimial solutions is guaranteed. For notational convenience, let w denote 









X 


w = 


Wy 




y 




w z 




z 



Definition 1 The design optimization problem of optimizing a quadratic function 
of several variables, including (shared) design variables as well as state and coupling 
variables, subject to linear constraints on these variables is called Quadratic Design 
Programming (QDP). QDP is a special type of multidisciplinary design optimization 
problem and can be formulated as follow: 

min w Qw + P w 

w z 

s.t. Aw < b 

Ew = d (18) 
y = Cw 
x = Xw 

Definition 2 An special class of QDP problem can be defined as follow: 



mm ^c i (fc)( 2l (fc)-y l (fc)-t i (fc)) 2 (19) 
i=l 

m 

(X r z r (k) + fl r yr (k)) < b 

r=l 

m 

s.t. j/j (k) = ^^bj r (fc) z r (k) , i = 1 , m 

r=l 

The next proposition presents convergence result for algorithm ( |17[ ) for design problem 
01 

Proposition 4 Let the sequence {zi(A;)} be generated by the collaborative design opti- 
mization algorithm (_?7| ) for the QDP problem \l£ty with the following weights, for all i 
and j 

Oij (k) = aij (k) - 2cti{k)c (k) bj (k) 2 

rn 

b'i (*) = 1 - X>i (*) 
r=l 

Also, let Weights Rule, Doubly Stochasticity, Connectivity, and Information Ex- 

m 

change Assumptions hold (cf. Assumptions 2, 3, 4, and 5) and, let the set Z = PI 
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be nonempty. Under Same Constraint Set Assumptions with the stepsize satisfying 
Sfc a k = 00 an d JZfc a k < 00 • Then, there exists an optimal point z* g stxc/i i/iaf 

lim «j (k) — z*\\ =0 for all i 

Proof The proof idea is straightforward. It is to show that the collaborative design 
optimization algorithm \17\ for QDP problem reduces to the distributed constrained 
optimization equation |2|. First, we calcualte the gradient of quadratic objective func- 
tion: 



where 



<k (*) = V,/(v,2) 
= V 



( k ) (*i ( fc ) ~ Vi ( fc ) - *i ( fc )) 

»=1 

m o~ 

(fc) (&• (fe) ^ (fe) - ^ (/c) 

■i=i 

m 

^c, (fc) (&• (fc) (fc) - ti (fc) 

i=l 

^c,(fc)V 2 (fcj(fc)*i(fc) -**(*))* 

i=l 
m 

^2 Cl (fc) &; (*) (&; (A) zi (fc) - *i 

i=l 

m 

b'i (fc) = 1 - ^6 rl (A) 



By substituting in equation (171 



(fc + 1) = P Si 



^2 a ij ( fe ) Z J ( fc ) - a i ( fc ) d i ( fc ) 
3 = 1 



3=1 



-a, (fc) ^2c, (fc) b'i (fc) (fc- (fc) z % (fc) - tj (fc) 



111 

Yl (( a «J ( fc ) - 2a i( k ) c j (fc) (fc) 2 ) Zj (k) ■ ■ ■ 
3=1 

..■-(2a l (fc) Cj (fc)&;(fc) 2 )t J (fc)) 
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we conclude 



(fc+1) 



St 



It I 



.7 = 1 



where 



ciij (k) = a,ij (fc) — 2cti(k)cj (k) bj (fc) 2 
Pi j {k) = 2a i (k)c J (k)b' J (k) 2 

Afterwards, the results are concluded by using proposition 

Remark 3 Every distributed design optimization problem without design variables up- 
dates (i.e., cti(k) = 0) reduces to a distributed average consensus problem. 

Remark 4 It is possible to guarantee convergence of a distributed design optimization 
problem both through sharing appropriate design variables (i.e., a^j (fc)) as well as 
coupling variables (i.e., bj (fc)). 

Proposition 5 Let the sequence {zj(fc)} be generated by the collaborative design opti- 



mization algorithm (11) for the QDP problem \19y with the following weights, for all i 
and j 



Oij (k) = ay (fc) - 2cti(k)cj (fc) bj (fc) 2 

m 

h (fc) = i - J2 b " ( fc ) 

r=l 

Also, let Weights Rule, Doubly Stochasticity, Connectivity, and Information Ex- 

m 

change Assumptions hold (cf. Assumptions 2, 3, 4, and 5) and, let the set Z = PI Z^ 

i=l 

be nonempty. Let the weight vectors 

dj (fc) = . . . , 1/m) for all iand k 

and the stepsize satisfying = oo and ~Y^k a "k ^ 00 ■ Then, the sequences {zj(fc)}, 

i = 1, ... ,m, converge to the same optimal point, i.e. 



lim Zj (fc) — z* =0 for some z* 6 Z* and all i 

k— >oo 

Proof Same as proposition Q. 

Proposition 6 Let the sequence {zj(fc)} be generated by the collaborative design op- 
timization algorithm 1(1 ify for the QDP problem (19) with ti (fc) = and the following 
weights, for all i and j 
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ay (k) = ciij (fc) — 2ai(k)cj (fe) bj (fe) 

TCI 

b'i (*) = i - x>™ (*) 

r=l 

A/so, Zei Weights Rule, Doubly Stochasticity, Connectivity, and Information Ex- 

ra 

change Assumptions hold (cf. Assumptions 2, 3, 4, o,nd 5) and, let the set Z = PI Z^ 

i=l 

be nonempty. We then have for some z £ Z and all i, 



lim | 

k— > oo 



(fe) 







Proof Same as proposition Q. The proof idea is to show that the collaborative de- 
sign optimization algorithm ( |17[ l for QDP problem reduces to the projected consensus 
algorithm when t t (k) = 0. 

Remark 5 Discussion of another proof of propositions Q, j5J), and (JgJ, provided by 
expanding the projection onto hyperplanes, omitted here due to space limitations (can 
be found in INooril I20T2I). 



Another special case that may be of interest is Quasi-seperable problems, (i.e., the 
problems where coupling objectives and constraints are not present). Further discus- 
sions of the different special cases will be omitted here in order to reserve more space 
and time for the discussion of the general problem. The interested reader is referred to 



Noori 2012 . 



5 PRELIMINARY NUMERICAL RESULTS 

The coupling among subspaces and state transitions are the main difference between 
distributed optimization and design optimization problems (see Figure j3j). In fact, 
guaranteed convergence of the general distributed design optimization problem in the 
presence of coupling dynamics is difficult. In special cases, it can be shown that this 
problem can be reduced to the distributed constrained optimization problem. In the 



following example, we consider an special class of QDP problem defined in ( 19 1 



Example 1 Let's consider the following design optimization problem P which is com- 
posed of four subproblems P\,P2,Pz and P4. The objectives contains both local and 
shared design variables, linked through coupling variables. For compliance with the 



definition of QDP problem, subproblems are defined according to (18 1. The structure 
of the problem is illustrated in Figure [5] 

Pi: min /1 = (zx (t) - y 2 (t)f + (z sl (t) - 10) 2 



+ (z s4 (t) - 10) 2 



s.t. 21 (i) + z s i (£) - Z S 4 (t) < 1 
XI (t) - z 3 (t) - 24 (t) = 
j/i (t) = xx (t) 
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Pj : min f r 



J s4 



22^2 .^3 £^2 

(£ 2 ,y 2 )e (X 2 ,F 2 ) 



y 3 



P 4 : min / 4 
(x 4 ,y 4 )e(X 4 ,r 4 ) 




P 3 : min / 3 
(£ 3 ,y 3 )E (X,^) 



Fig. 5 Example 1 



P 2 : min / 2 = (z 2 (t) + yi (t)f + {z sl (i) - 4) 2 

z 2 , z sl ; Z s2 

+ (ZS2 W-4) 2 
S.t. 2 2 (£) - Zsl (t) ~ Z s2 (t) < 1 
X 2 (t) - Z 2 (t) = 
VI if) = X 2 (t) 



P 3 : min f 3 = (z 3 (t) - y 4 (t)f + (z s2 (t) + 2) 2 
+ ( 2s3 (t) - 5) 2 

S.t. Z3 (*) + 2s2 (*) +2 S 3 (*) > -1 
^3 (*) - Z S 2 (0 = 

2/3 (i) = x 3 (t) 



Pi 



mm 

z 4j z s3 ) 2 s4 



(Z4 (t) + y 3 (t)) 2 + (z s3 (t) + 10) 2 



+ {z s4 (t) + 10) 2 

S.t. Z 4 (t) ~ Z s3 (t) + Z s4 (t) > -1 
X 4 (t) - z sl (t) = 
1/4 (*) = X 4 (t) 

Each quadratic objective function are constrained on a one-dimensional locus (i.e., 
a line) and within a half-space wit h a hyperplane. T he early implementation of the 

in a 32-bit environment. The 



MathWorks 



2010 



algorithm is done on MATLAB® 
testbed environment consists of 4 Intel Pentium IV. 2,5GHz and 1GB RAM worksta- 
tions running Microsoft Windows XP Professional. The PCs are interconnected by a 
100Mbit Ethernet LAN setted-up as a single collision domain. 

The test results are presented in table [l] The figure [6] shows the performance 
of the algorithm that uses the following settings; optimization coefficienta pt = 
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Table 1 Comparision of simulation results 



Var. 


AIO method 


CMDO method 




21 


-6.000 


-5.999 


22 


-6.000 


-6.000 


23 


7.000 


7.000 


24 


-0.999 


-1.000 


Z S 1 


7.000 


7.000 


Zs2 


0.999 


1.000 


2 S 3 


-2.499 


-2.499 


2 S 4 


0.000 


0.000 




Fig. 6 Convergence of the solutions 



consensus coefncienta CO n = and iteration n = 10000. As depicted in Figure [fs] 
while convergence to the optimal solution is quickly achieved (about 100 iterations) but 
approaching to accurate result is slow (10000 iterations). This feature of the algorithm 
could be improved in several ways; amongst them are proper network weight design or 
proper selection of optimization and consensus factors. 



6 CONCLUSION 



In this paper, a distributed computation framework for multi-objective multidisci- 
plinary design optimization problems is proposed. The corresponding coordination 
strategy is collaborative and concurrent, which make it suitable for real-world de- 
sign problems. It is also shown that distributed constrained optimization problem is an 
special case of collaborative multidisciplinary design optimization problem. We also in- 
vestigate an important class of design optimization problems, called QDP problem. By 
using existing results, we established convergence of an special case of QDP problem. 
Finally, the paper highlights challenging areas in which research is required to allow 
us to utilize the full potential of distributed optimization methods in multidisciplinary 
design optimization in the future. 
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